TITLE OF THE INVENTION 
IMAGE PROCESSING APPARATUS AND METHOD 



FIELD OF THE INVENTION 
5 The present invention relates to an image 

processing apparatus and method and, more particularly, 
to image processing of embedding a different kind of 
information in an image as added information. 

10 BACKGROUND OF THE INVENTION 

Apparatuses such as printers and copying machines 
that print or form color images are making great 
progresses in terms of both performance and penetration. 
In addition, to print or form full-color images, many 

15 schemes including silver halide scheme, thermal 

printing, electrophotography, electrostatic printing, 
and Inkjet printing have been developed, and 
high-quality full-color images can be obtained. 
However, a new problem rises under these circumstances. 

20 That is, bank notes and securities can easily be 

forged using a full-color image printing apparatus. 
For this reason, full-color image printing apparatuses 
must have a function of preventing forgery. Recent 
full-color image printing apparatuses have various 

25 forgery preventing functions. The most general scheme 
of implementing a forgery preventing function is 
so-called pattern tracking in which in printing an 
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image, a periodic dot pattern representing the machine 
number of the apparatus or the like is printed on the 
printing paper sheet. If a forged bank note or the 
like is found, the machine number is detected from the 
5 dot pattern to specify the apparatus used for forgery. 
The dot pattern is printed on all printing paper sheets 
to be output from the apparatus and is therefore 
generally printed using the most unnoticeable color, 
i.e., yellow. 

10 In an apparatus for adding a different kind of 

information except an image by printing such a dot 
pattern on a printing paper sheet together with an 
image (to be expressed as "embedding a dot pattern in 
an image" hereinafter) , the influence of the embedded 

15 dot pattern on the image and the accuracy of the 

different kind of information detected from the printed 
image depend on the shape of an on-dot pattern (to be 
described later) as a component of the embedded dot 
pattern. That is, use of an on-dot pattern that mainly 

20 aims at suppressing any degradation in image quality 

sacrifices accurate information detection. Conversely, 
use of an on-dot pattern that aims at improving the 
detection accuracy largely degrades the image quality. 

25 SUMMARY OF THE INVENTION 

The present invention has been made to solve the 
above problems individually or altogether, and has as 
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its object to provide dot pattern embedding that 
suppresses any degradation in image quality and ensures 
high detection accuracy. 

In order to achieve the above object, according 
5 to a preferred aspect of the present invention, an 

image processing apparatus for embedding a dot pattern 
which indicates added information, comprising: a 
halftone processor, arranged to execute error diffusion 
for an image; a determiner, arranged to determine a 

10 component of the dot pattern to be embedded on the 

basis of the image that has undergone error diffusion; 
and an embedding section, arranged to embed the dot 
pattern in the image that has undergone error diffusion 
using the determined component is disclosed. 

15 Also, an image processing method of embedding a 

dot pattern which indicates added information, 
comprising the steps of: executing error diffusion for 
an image; determining a component of the dot pattern to 
be embedded on the basis of the image that has 

20 undergone error diffusion; and embedding the dot 

pattern in the image that has undergone error diffusion 
using the determined component is disclosed. 

Other features and advantages of the present 
invention will be apparent from the following 

25 description taken in conjunction with the accompanying 
drawings, in which like reference characters designate 
the same or similar parts throughout the figures thereof. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is a block diagram showing added 
information embedding processing; 
5 Fig. 2 is a view showing a specific dot pattern 

added to a yellow plane; 

Fig. 3 is a view showing an on-dot pattern; 
Fig. 4 is an enlarged view of Fig. 2; 
Fig. 5 is a flow chart showing the flow of 
10 processing in an added information superposing sections- 
Fig. 6 is a view showing on-dot patterns 
corresponding to on-dot data; 

Fig. 7 is a view for explaining density 
maintaining processing; 
15 Fig. 8A is a view showing a color component image 

after pseudo-halftoning; 

Fig. 8B is a view showing the two-dimensional 
Fourier power spectrum of the color component image 
shown in Fig. 8A; 
20 Fig. 8C is a graph that plots the distribution of 

a power P in the radial direction of the circle shown 
in Fig. 8B; 

Fig. 9A is a view showing a color component image 
obtained by adding the on-dot shown in Fig. 6(b), which 
2 5 is formed from 3x3 dots, to the color image component 
shown in Fig. 8A; 

Fig. 9B is a view showing the two-dimensional 
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Fourier power spectrum of the color component image 
shown in Fig. 9A; 

Fig. 9C is a graph that plots the distribution of 
the power P in the radial direction of the circle shown 
5 in Fig. 9B; 

Fig. 10 is a flow chart for explaining on-dot 
pattern shape determination processing; 

Fig. 11 is a view showing on-dot patterns; 

Fig. 12 is a view showing states wherein dot 
10 patterns are embedded in the color component image 
shown in Fig. 8A using the on-dot patterns shown in 
Fig. 11; 

Fig. 13 is a graph showing the two-dimensional 
Fourier power spectra of the color component images 
15 shown in Fig. 12; 

Fig. 14 is a view showing a general error 
diffusion filter used for pseudo-halftoning; 

Figs. ISA to 15C are views showing color 
component images after error diffusion; 
20 Figs. 16A to 16C are views showing images 

obtained by embedding dot patterns in the color 
component images shown in Figs. 15A to 15C without 
considering the texture; 

Fig. 17 is a view showing on-dot patterns to be 
25 added to the color component images shown in Figs. 15A 
to 15C in the second embodiment; 

Figs. 18A to 18C are views showing color 
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component images in which dot patterns are embedded 
using the on-dot patterns shown in Fig. 17; 

Fig. 19 is a view showing on-dot patterns to be 
added to the color component images shown in Figs. 15A 
5 to 15C in the third embodiment; and 

Figs. 20A to 20C are views showing color 
component images in which dot patterns are embedded 
using the on-dot patterns shown in Fig. 19. 

(.a 

% 10 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

|:S Image processing according to embodiments of the 

i: 2 present inventions will be described below in detail 

with reference to the accompanying drawings. 
[First Embodiment ] 
15 [Arrangement] 

The outline of information (to be referred to as 
"added information" hereinafter) printing (embedding) 
processing using a dot pattern will be described. 
Fig. 1 is a block diagram showing added 
20 information embedding processing. Referring to Fig. 1, 
an input image signal represented by R, G, and B color 
components is converted into four color components, 
i.e., C (cyan), M (magenta), Y (yellow), and K (black) 
by a color conversion section 101. Each color 
25 component is subjected to correction processing by a 
multi-correction processing section 102. 

Next, a pseudo-halftoning section 103 executes 
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pseudo-halftoning using systematic dithering or error 
diffusion. On the other hand, an added information 
generation section 104 generates added information on 
the basis of the ID of the printer main foody or user 
information. The generated added information is 
superposed on the Y color component by an added 
information superposing section 105. The C, M, Y, and 
K color components are input to a printer engine 10 6. 
Then, an image 107 in which the added information other 
than the image information is embedded is printed. 

Fig. 2 is a view showing a specific dot pattern 
added to a yellow plane. 

Referring to Fig. 2, an A x B inch area 201 
indicated by broken lines is called an "information 
area". Added information is expressed by a dot pattern 
in the area 201. The information areas continuously 
and periodically exist, as shown in Fig. 2. Each pixel 
indicated by symbol (■) in Fig. 2 is formed from 3x3 
dots shown in Fig. 3. An enlarged view of Fig. 2 is 
shown in Fig. 4. 

Symbol (■) shown in Fig. 4 indicates a dot (to 
be referred to as an "on-dot" hereinafter) of a dot 
pattern printed on a printing paper sheet with yellow 
ink or yellow toner. 

The added information superposing section 105 
shown in Fig. 1 will be described here in more detail. 
Fig. 5 is a flow chart showing the flow of processing 



in the added information superposing section 105. 

In step S501, a plurality of kinds of on-dot data 
511 prepared in advance (stored in a memory) are loaded. 
Fig. 6 is a view showing on-dot patterns corresponding 
5 to the on-dot data 511. Fig. 6(a) shows an on-dot 
pattern for a low density, Fig. 6(b) shows an on-dot 
pattern for a medium density, and Fig. 6(c) shows an 
on-dot pattern for a high density. This description 
will be done assuming that the printer has a function 
!;i 10 of selectively printing a high-density dot (to be 

!,£ referred to as a "dark dot" hereinafter) or a 

: *J low-density dot (to be referred to as a "light dot" 

"P hereinafter) . Hence, a low- or medium-density on-dot 

I;* is printed by a light dot, and a high-density on-dot is 

ft! 15 printed by a dark dot. 

Q In step S502, added information 512 is loaded. 

In step S503, the density of a color component image 
near the on-dot adding position is estimated, and a 
suitable on-dot pattern is selected from the three 

20 on-dot patterns. In step S504, density maintaining 

processing is executed to keep the density of the color 
component image near the on-dot adding position 
unchanged from that before addition of the on-dot, and 
the on-dot is added. Steps S503 and S504 are repeated 

25 until it is determined in step S505 that the on-dots 
are added to all on-dot adding positions. 

The density maintaining processing in step S504 
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will be described in more detail. 

Fig. 7 is a view for explaining the density 
maintaining processing. Fig. 7(a) shows a color 
component image which has undergone pseudo-halftoning. 
5 Fig. 7(b) shows a color component image obtained by 
adding on-dots (Fig. 6(b)) without executing the 
density maintaining processing. Referring to Fig. 7(b), 
the density increases near the added on-dots. Hence, 
even when the unnoticeable yellow color is used, 

10 unnaturally visible dots may be formed to degrade the 
quality of the output image. 

To suppress formation of unnatural dots, dots 
that are present in Fig. 7(a) are set off (dots are not 
formed) near the on-dot adding positions to prevent the 

15 number of formed dots from changing after addition of 
the on-dots. With this operation (density maintaining 
processing) , the color component image shown in 
Fig. 7 (c) is obtained, in which formation of unnatural 
dots is suppressed as compared to Fig. 7(b). 

20 [Optimizing On-Dot Pattern] 

In the first embodiment, an optimum on-dot 
pattern shape is determined by evaluating the influence 
of on-dot addition on the image quality in the 
frequency domain. 

25 Fig. 8A is a view showing a color component image 

obtained by executing pseudo-halftoning using an error 
diffusion filter for a color component image which has 
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a relatively low and uniform density. Fig. 8B is a 
view showing the two-dimensional Fourier power spectrum 
of the color component image shown in Fig. 8A. In 
other words, Figs. 8A and 8B show the characteristic of 
the error diffusion filter. 

Referring to Fig. 8B, the abscissa represents a 
horizontal frequency fx, the ordinate represents a 
vertical frequency fy, and the origin represents a DC 
component. The area in the circle with a radius fc is 
an area having a relatively low power. The area 
outside the circle is an area having a relatively high 
power . 

Fig. 8C is a graph that plots the distribution of 
a power P in the radial direction of the circle shown 
in Fig. 8B. Referring to Fig. 8C, the ordinate 
represents the power P, the abscissa represents a 
radial frequency fr, and the origin represents the 
center of Fig. 8B. As is apparent from Fig. 8C, the 
error diffusion filter has a high-pass characteristic 
using the frequency fc as a cutoff frequency. 

Fig. 9A is a view showing a color component image 
obtained by adding the on-dot shown in Fig. 6(b), which 
is formed from 3x3 dots, to the color image component 
shown in Fig. 8A. Note that the above-described 
density maintaining processing is executed in adding 
the on-dots. 

Fig. 9B is a view showing the two-dimensional 



Fourier power spectrum of the color component image 
shown in Fig. 9A. As is apparent from Fig. 9B, when 
the on-dots are added, the power P increases in the 
low-frequency domain. Fig. 9C is a graph that plots 
the distribution of the power P as in Fig. 8C. The 
power P mainly increases in the low-frequency domain 
probably because the on-dot pattern shown in Fig. 6(b) 
has oblique dot connection. 

In embedding a dot pattern in an image as a code 
that expresses certain information and extracting the 
information from the printed image using an image 
reading apparatus such as a scanner, in order to 
increase the information detection accuracy while 
suppressing the degradation in image quality due to dot 
pattern embedding, the density of dots of the dot 
pattern must be increased to some extent. However, if 
the dot pattern as shown in Fig. 9A is used, the power 
in the low-frequency domain increases because the dot 
density of the dot pattern to be embedded is higher 
than the pixel density of the image near the on-dot 
adding position. The human visual characteristic is 
supposed to have a high sensitivity to a low-frequency 
component. Hence, any unnecessary increase in power in 
the low-frequency domain considerably degrades the 
image quality. 

To the contrary, when a dot pattern that 
generates no unnecessary increase in power in the 



low-frequency domain is embedded, the image quality 
hardly degrades because of the human visual 
characteristic. In the first embodiment , an 
appropriate on-dot pattern shape that causes no 
unnecessary increase in power in the low-frequency 
domain of the color component image as shown in Fig. 8A 
is determined by on-dot pattern shape determination 
processing (to be described later) . 

Fig. 10 is a flow chart for explaining on-dot 
pattern shape determination processing according to the 
first embodiment. This processing is executed in, e.g., 
step S503 shown in Fig. 5 where an optimum pattern is 
selected. 

In step S1001, N (> 1) types of dot patterns are 
prepared as candidates for an appropriate on-dot 
pattern. That is, on-dot pattern candidates are 
selected from the on-dot patterns loaded in step S501. 
As the on-dot pattern candidates, several (N) types of 
on-dot patterns are selected on the basis of the 
above-described density of the color component image 
near the adding position. For the following 
description, assume that three different on-dot 
patterns Pi to P3 shown in Fig. 11 is selected. 

In step S1002, a counter n = 1 is set. In step 
S1003, a dot pattern is embedded in the color component 
image using an on-dot pattern Pn corresponding to the 
counter n. In step S1004, the image having the 
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embedded dot pattern is converted into a frequency 
domain using Fourier transform or the like, and the 
power spectrum of the image is obtained. 

Figs. 12 is a view showing states wherein dot 
5 patterns are embedded in the color component image 
shown in Fig. 8A using the three different on-dot 
patterns PI to P3 shown in Fig. 11. Let i(x,y) be the 
image before the dot pattern is embedded, i (x,y) be 
[:4 the image having the dot pattern embedded using the 

:•' 

:. :: 3 

i;3 10 on-dot pattern Pn, and I(fx,fy) and I (fx,fy) be the 

images obtained by converting these images into 
is; frequency domains. Then, power spectra S(fx,fy) and 

S (fx,fy) of the images are given by 
t S(fx,fy) = |I(fx,fy) | 2 ... (1.1) 

J 15 S pn (fx,fy) = |I pn (fx,fy) | 2 ...(1.2) 

13 Next, in step S1005, an evaluation value En that 

evaluates the degradation in image quality when the dot 
pattern is embedded by the on-dot pattern Pn is 
obtained using the resultant power spectra. 
20 En = X fr<fma JS pn (fx, fy) - S(fx,fy)| ...(1.3) 

for fr = J~ (fx + fy 2 ) 

That is, the evaluation value En is obtained by 
calculating the difference between the power spectra 
obtained by equations (1.1) and (1.2) and totalizing 
25 the difference values within the range of fr < fmax. 
The value fmax represents the upper limit of the 
frequency range used to calculate the difference value. 
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The cutoff frequency fc or any other arbitrary 
frequency can be used as fmax. 

Processing in steps S1003 to S1005 is repeated, 
and the value N is incremented in step S1007 until it 
5 is determined in step S1006 that n = N. 

Fig. 13 is a graph showing the two-dimensional 
Fourier power spectra of the color component images 
shown in Fig. 12. When the images having dot patterns 
embedded by the on-dot patterns PI to P3 have frequency 
10 characteristics as shown in Fig. 13, evaluation values 
El to E3 obtained by equation (1.3) have a relationship 
given by 

El < E2 < E3 ... (1.4) 

In step S1008, an appropriate on-dot pattern is 

15 determined on the basis of the evaluation value En. To 
suppress any degradation in image quality due to dot 
pattern embedding, the on-dot pattern PI having the 
smallest evaluation value En can be optimally used. 
However, that the evaluation value En is small also 

20 means that the shape of the on-dot pattern Pn is 

similar to the dot layout of the image. Hence, it is 
more difficult to detect the added information from the 
printed image by a reading apparatus. To prevent this, 
in the first embodiment, a lower limit value Emin 1010 

25 is prepared. Any on-dot pattern that satisfies En < 
Emin is not employed. The lower limit value Emin is 
preset in accordance with the characteristic of the 
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error diffusion filter or the density state of the 
image . 

For example, if the evaluation value El of the 
on-dot pattern PI is smaller than the lower limit value 
Emin, the on-dot pattern PI is not employed. As a 
consequence, the on-dot pattern P2 is employed as an 
appropriate dot pattern for the color component image 
shown in Fig. 8A. If El < E2 < Emin, the on-dot 
pattern P3 is employed. 

As described above, in adding a dot pattern to an 
image, if an on-dot pattern is selected such that an 
increase in power in the low-frequency domain is 
minimized, the dot pattern can be embedded with less 
degradation in image quality. 

Processing in an area having a relatively low 
image density has been described above. With the same 
processing as described above, an appropriate on-dot 
pattern can be determined even for an area having a 
high image density. 
[Second Embodiment] 

Image processing according to the second 
embodiment of the present invention will be described 
below. The same reference numerals as in the first 
embodiment denote almost the same components in the 
second embodiment, and a detailed description thereof 
will be omitted. 

In the first embodiment, a method of evaluating 
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the influence of dot pattern embedding on an image in 
the frequency domain and determining an appropriate 
on-dot pattern shape on the basis of the evaluation 
result has been described. In the second method, a 
5 method of determining an appropriate on-dot pattern 

shape on a real space in accordance with the texture of 
an image will be described. 

Fig. 14 is a view showing a general error 
diffusion filter used for pseudo-halftoning. Symbol 

10 (•) indicates a pixel to be quantized. Peripheral 
pixels with error diffusion coefficients a to d are 
pixels to which an error will be diffused (to be 
referred to as "error diffusion pixels" hereinafter) . 

In error diffusion, the difference value between 

15 a pixel value quantized at a given threshold value and 
a pixel value before quantization is weighted by the 
error diffusion coefficients a to d and distributed 
(added) to error diffusion pixels. Generally, when the 
error diffusion range is wide, generated dots are 

20 uniformly distributed. However, error diffusion in a 
very wide range is impossible from the viewpoint of 
processing speed. An error is distributed to about 
four pixels, as shown in Fig. 14. For this reason, the 
image after error diffusion has a chain-like or 

25 peculiar texture. 

Figs. ISA to 15C are views showing color 
component images after error diffusion. Fig. ISA shows 
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an area having a relatively low density after error 
diffusion in which no peculiar texture is observed. On 
the other hand, a diagonal texture downward from the 
upper left is observed in Fig. 15B (medium density 
5 area) . A texture with vertically connected dots is 
observed in Fig. 15C (high density area) . When dot 
patterns are embedded in the color component images 
shown in Figs. 15A to 15C without considering the 
l<A texture, images shown in Figs. 16A to 16C are obtained. 

I!!? 10 In the second embodiment, a dot pattern is 

f.JJ embedded in consideration of texture formation by error 

2 diffusion. In the second embodiment as well, an error 

"F diffusion filter is used for pseudo-halftoning, and the 

M a printer is assumed to be able to selectively print a 

i y 

!'U 15 dark dot or a light dot. 

□ Figs. 17 is a view showing on-dot patterns to be 

as: 

added to the color component images shown in Figs. 15A 
to 15C in the second embodiment. 

Since no peculiar texture is observed in the 

20 color component image shown in Fig. 15A, the on-dot 

pattern shown in Fig. 17(a) is used. Since a diagonal 
texture downward from the upper left is observed in the 
color component image shown in Fig. 15B, the on-dot 
pattern for a medium density shown in Fig. 17(b), which 

25 is formed from three light dots laid out in the same 
direction as that of the texture, is used. Since a 
vertical texture is observed in the color component 
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image shown in Fig. 15C, the on-dot pattern for a high 
density shown in Fig. 17(c), which is formed from three 
dark dots laid out in the same direction as that of the 
texture, is used. 
5 Figs. 18A to 18C are views showing color 

component images in which dot patterns are embedded 
using the on-dot patterns shown in Fig. 17. As shown 
in Figs. 18A to 18C, when an on-dot pattern having a 

M= shape corresponding to the direction of texture formed 

w 

rj 10 by error diffusion is determined and added to each 

density area, the dot pattern can be embedded in the 

M 

ill image while suppressing the influence on the image 

SB 

quality, 
j!^ [Third Embodiment] 

I™ 15 Image processing according to the third 

;*J embodiment of the present invention will be described 

below. The same reference numerals as in the first 
embodiment denote almost the same components in the 
third embodiment, and a detailed description thereof 
20 will be omitted. 

In the second embodiment, an example in which an 
on-dot pattern corresponding to the direction of 
texture is used in consideration of a texture formed by 
error diffusion has been described. According to the 
25 second embodiment, a dot pattern can be embedded in an 
image while suppressing the influence on the image 
quality. However, it may be difficult to detect the 
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added information from a printed image using a reading 
apparatus because the direction of texture matches or 
resembles the direction of on-dot pattern. 

In the third embodiment, on-dot patterns shown in 
5 Fig. 19 are employed. The on-dot pattern shown in 

Fig. 19(a) is used for the low-density image shown in 
Fig. 15A. The on-dot pattern shown in Fig. 19(b) is 
used for the medium-density image shown in Fig. 15B. 
u The on-dot pattern shown in Fig. 19(c) is used for the 

!^ 10 high-density image shown in Fig. 15C. 

Figs. 20A to 20C are views showing color 
,2: component images in which dot patterns are embedded 

"I* using the on-dot patterns shown in Fig. 19. As shown 

j;* in Figs. 20A to 20C, when an on-dot pattern almost 

! S U 15 perpendicular to the direction of texture formed by 

Q error diffusion is determined and added to each density 

area, the dot pattern can be embedded in the image 
while suppressing the influence on the image quality to 
almost the intermediate level between the case wherein 
20 the texture is not taken into consideration (Figs. 16A 
to 16C) and the second embodiment (Figs. 18A to 18C and 
increasing the added information detection accuracy. 

Processing of the second and third embodiments is 
executed in, e.g., step S503 shown in Fig. 5 where an 
25 optimum pattern is selected. In addition, the 

direction of texture can be known by known filtering, 
and a detailed description thereof will be omitted. 
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When a vertical (or horizontal) texture is formed 
in a medium-density area, each dot of the on-dot 
pattern shown in Fig. 17(c) (or Fig. 19(c)) is changed 
to a light dot, and a resultant dot pattern is employed. 
When a diagonal texture is formed in a high-density 
area, each dot of the on-dot pattern shown in 
Fig. 17(b) (or Fig. 19(b)) is changed to a dark dot, 
and a resultant dot pattern is employed. 

A diagonal texture downward from the upper right 
or a horizontal texture may be formed depending on the 
structure of the error diffusion filter or processing. 
In that case, the on-dot patterns shown in Fig. 17 and 
Fig. 19 are adaptively used. 

According to the above-described embodiments, in 
an apparatus for embedding a dot pattern which 
specifies the apparatus or user in an image after 
pseudo-halftoning in order to prevent forgery of a 
specific original such as a bank note or securities 
using a color image printing apparatus (printer or 
copying machine), the on-dot pattern (or its shape) to 
be used to embed a dot pattern is determined on the 
basis of the power spectrum in a frequency domain after 
dot pattern embedding or a texture formed by 
pseudo-halftoning, thereby multiplying (superposing) 
added information with less visual sense of 
incompatibility (with less degradation in image 
quality) . 
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Additionally, when an on-dot pattern is selected 
and used in consideration of the direction of texture, 
the added information detection accuracy can be 
increased while suppressing the degradation in image 
quality. 

As many apparently widely different embodiments of 
the present invention can be made without departing from 
the spirit and scope thereof, it is to be understood 
that the invention is not limited to the specific 
embodiments thereof except as defined in the appended 
claims . 



